The invention relates to the formation of digital images from auxiliary graphical elements which form auxiliary graphical images or sub-images (xe2x80x9csub-picturesxe2x80x9d). In particular, the invention relates to the decoding of these sub-images which are intended to be inset into a predetermined area of a display screen and superimposed on main images.
Images to be decoded, for example, according to the MPEG standards (xe2x80x9cMotion Pictures Experts Groupxe2x80x9d) are stored as compressed data on, for example, a multifunction digital disc (DVD: xe2x80x9cDigital Versatile Diskxe2x80x9d). Furthermore, auxiliary images or sub-images which result from auxiliary data or sub-image data (sub-picture data) are stored in a compressed manner on a such a disk.
One of the uses of these auxiliary graphical images or sub-images relates to the display of menus on the screen of a television. Also, these images can be used to inset small moving sequences on the main images depicted on the screen of a television set. Additionally, these images can be used for the depicting of texts of songs in karaoke applications.
A current method used for forming auxiliary graphical images includes decoding the auxiliary graphical data, compressed and stored on the digital disk, with software to store the decoded auxiliary graphical images in a dynamic memory. In this way, the auxiliary graphical images are ready to be extracted from the memory, as many times as necessary, to be displayed each time pixel by pixel. However, such a method requires a substantial memory size, especially when the size of the auxiliary graphical images is substantial.
An object of this invention is to solve this problem by proposing an approach for the decoding and forming of these auxiliary graphical images, and for the forming of the images resulting from being superimposed on the stream of main images. The invention therefore proposes a process for forming digital images resulting from auxiliary graphical elements inset on the main images.
Generally, the invention includes a process which comprises the receipt of at least one auxiliary stream comprising a set of compressed graphical data corresponding to graphical elements forming at least one auxiliary graphical image intended to be inset into a predetermined area on a display screen. The auxiliary graphical image, superimposed on a main image, is displayed at the occurrence of a predetermined initial main image and for a predetermined elementary duration. Furthermore, the process comprises the storage of the auxiliary stream in a memory, and successive decodings of the compressed graphical data. These successive decodings are respectively performed during the display of the successive main images from the display of the initial main image up to the expiration of the predetermined elementary duration. Also, these successive decodings are performed to deliver in the course of each decoding, at the rate of display of the pixels on the screen and for each pixel of the auxiliary image, luminance and chrominance values as well as a mixing coefficient. The process also comprises the pixel-by-pixel mixing, in the predetermined screen area, of the chrominance and luminance values corresponding to the current main image being displayed and to the decoded auxiliary graphical image, having the mixing coefficients.
In other words, the process according to the invention provides for an on the fly decoding of the compressed graphical data of a graphical image, during the display of a main image (in practice and for example, during the display of each frame of the main image). Furthermore, the process provides for the redecoding of the same data corresponding to the same graphical image for each main image, i.e. in the course of the display of each main image, and for as long as necessary. Thus, for example, if an auxiliary graphical image needs to be inset on the display screen superimposed on an initial main image and superimposed on the next 99 main images, the compressed graphical data corresponding to each frame of this auxiliary image will be decoded 100 times in succession. Each decoding taking place in real time during the display of the current main image so that the mixer receives in a synchronized manner, for each pixel, the luminance and chrominance values of the main image intended to be displayed. And, the mixer receives the luminance and chrominance values of the decoded auxiliary graphical image likewise intended to be displayed and superimposed on this main image being displayed.
The invention therefore has the notable advantage of reducing the necessary memory size as well as the memory passband used because the auxiliary graphical data of each graphical image may be kept in the memory in a compressed form. The decoding of this data does not require storage of the decoded auxiliary graphical image in the memory before its display.
The auxiliary stream generally comprises a header, the compressed graphical data, and at least one set of parametrization data, making it possible to parametrize the decoding of the graphical image. Moreover, according to one embodiment of the invention in which the main images are displayed line by line in two successive frames corresponding to lines of different parities, a process is performed for analyzing the parametrization data between each display of a frame. This being so, it would also be possible in another embodiment of the invention, to perform this process for analyzing the parametrization data during each line flyback. In fact, generally, the process for analyzing the parametrization data is performed during a period when there is no pixel display on the screen, so as not to disturb this display.
The set of compressed graphical data can correspond to several successive auxiliary graphical images each intended to be superimposed in succession with the main images for a respective elementary duration. The auxiliary stream thus comprises several sets of parametrization data each making it possible to parametrize the decoding of an auxiliary graphical image. And, so long as the elementary duration corresponding to an auxiliary graphical image has not elapsed, the set of parametrization data corresponding to this image is analyzed between each display of a frame. Then the compressed graphical data of this image is decoded on the basis of the parametrization data. And, when the corresponding elementary duration has elapsed, there is an automatic transfer to the analysis of the set of parametrization data corresponding to the next auxiliary graphical image. Then compressed graphical data of this next image is decoded on the basis of these new parametrization data.
According to a particular embodiment of the invention, on receiving the auxiliary stream, a temporal counter is initialized to a predetermined initial value (for example the time of display of the first auxiliary graphical image). Each set of parametrization data comprises a temporal indication, linked to the predetermined initial value (for example the start time of display of each of these auxiliary graphical images), and representative of the start instant of the first decoding of the corresponding auxiliary graphical image. Before commencing the decoding of the compressed data corresponding to the first auxiliary graphical image, the temporal indication of the next auxiliary graphical image is stored in a register and the successive decodings of the auxiliary graphical image are performed. These decodings are parametrized by the corresponding set of parametrization data so long as the value of the counter is less than the content of the register. And, when the value of the counter becomes greater than or equal to the content of the register, then it is time to carry out the decoding and displaying of the next auxiliary graphical image.
According to one embodiment of the invention, the processing for analyzing a set of parametrization data comprises the storing of different groups of color/mixing coefficient pairs in different respective elementary memories (look-up tables). The number of pairs per group is less than a first predetermined number (for example four when the tables are addressed by 2 bits). Furthermore, the number of groups is less than or equal to a second predetermined number (for example eight, corresponding to a maximum of eight different regions per image line). The line-by-line decoding of each auxiliary graphical image thus comprises successive addressing by the decoded graphical data, of the various elementary memories designated in succession by designation data contained in the set of parametrization data.
In other words, by way of example, once several look-up tables have been programmed during the process for analyzing the parametrization data, a first look-up table is addressed in respect of the first pixels of the line. Then, as a function of a designation datum corresponding to the size of the region (for example, the number of pixels), there is a switch, after formulating this number of pixels, over to another look-up table which will be addressed on the basis of the next decoded graphical data, and so on and so forth.
The subject of the invention is also a system for forming digital images resulting from auxiliary graphical elements inset on main images. According to a general characteristic of the invention, this image forming system comprises a receiver for receiving a main stream comprising main digital image compressed data, and at least one auxiliary stream comprising a set of compressed graphical data corresponding to graphical elements forming at least one graphical image. Such a graphical image intended to be inset into a predetermined area of a display screen, from the occurrence of an initial main image and for a predetermined elementary duration, and superimposed on the main images. The system also includes a memory for storing the main stream and the auxiliary stream, and a main decoder (for example an MPEG decoder if the main images are compressed according to an MPEG standard). This main decoder being able to decode the main stream of compressed data and to deliver for each pixel of a decoded main image, luminance and chrominance values.
Furthermore, the system also comprises a device for forming the resulting images comprising a decoder (e.g. an auxiliary decoder, sub-image decoder or sub-picture decoder) able to perform successive decodings of the compressed graphical data contained in the memory. These successive decodings are performed respectively during the display of the successive main images, from the display of the initial main image up to the expiration of the predetermined elementary duration. These successive decodings are performed to deliver in the course of each decoding, at the rate of display of the pixels on the screen and for each pixel of the auxiliary image, luminance and chrominance values as well as a mixing coefficient. The image forming device further comprises a mixer able to perform the pixel-by-pixel mixing in the predetermined screen area of the chrominance and luminance values corresponding to the current main image and to the decoded auxiliary graphical image, with respect to the mixing coefficients.
According to one embodiment of the invention, in which the auxiliary stream comprises a header, the compressed graphical data, and at least one set of parametrization data making it possible to parametrize the decoding of the graphical image, the auxiliary decoder (sub-image decoder) comprises a first FIFO type memory (first in-first out) able to store the compressed graphical data, and a second FIFO type memory able to store the parametrization data. Moreover, the main images are displayed line by line and in two successive frames corresponding to lines of different parities. The auxiliary decoder comprises an analyzer able, between each displaying of a frame, to deliver the parametrization data to the second memory and to perform a process for analyzing these parametrization data. The auxiliary decoder also comprises a processor (comprising for example a variable-length decoder (VLD) and a zero-run decoder (RLD)) able to decode the compressed graphical data stored in the first memory, on the basis of the parametrization data. And, the processor is able to deliver to the mixer, for each pixel of the auxiliary graphical image, the luminance and chrominance values, as well as a mixing coefficient.
When the set of compressed graphical data corresponds to several successive auxiliary graphical images intended to be superimposed in succession on the main images for a respective elementary duration, the auxiliary stream comprises several sets of parametrization data each making it possible to parametrize the decoding of an auxiliary graphical image. And, so long as the elementary duration corresponding to an auxiliary graphical image has not elapsed, the analyzer delivers to the second memory, between each displaying of a frame, the corresponding parametrization data and performs a process for analyzing these parametrization data. The processor decodes the compressed graphical data stored in the first memory, on the basis of the parametrization data, and when the corresponding elementary duration has elapsed, the analyzer delivers to the second memory, between each displaying of a frame, the parametrization data of the next auxiliary graphical image and performs a process for analyzing these new parametrization data. The processor then decodes the compressed graphical data stored in the first memory and corresponding to this next auxiliary graphical image, on the basis of these new parametrization data.
According to one embodiment of the invention, the analyzer comprises a temporal counter and controller able to initialize the temporal counter to a predetermined initial value. Each parametrization datum comprises a temporal indication, linked to the predetermined initial value, and representative of the start instant of the first decoding of the corresponding auxiliary graphical image. The analyzer is thus able to store, in a register, the temporal indication of the next auxiliary graphical image. The processor performs the successive decodings of the current auxiliary graphical images, these decodings being parametrized by the corresponding set of parametrization data so long as the value of the counter is less than the content of the register.
According to one embodiment of the invention, the auxiliary decoder (sub-image decoder) comprises elementary memories (look-up tables). The analyzer is able to store respectively in at least some of these elementary memories, different groups of color/mixing coefficient pairs. The number of pairs per group is less than a first predetermined number, while the number of elementary memories is less than or equal to a second predetermined number (for example 8). The processor comprises an auxiliary decoder block (VLD circuit and RLD circuit) able to decode the compressed graphical data and to address the elementary memories with the decoded graphical data, as well as a multiplexer whose inputs are linked to the outputs of the elementary memories. The auxiliary decoder also comprises a controller of the multiplexer which is able to select the inputs of the multiplexer as a function of designation data contained in the parametrization data, these designation data making it possible to select one of the elementary memories (look-up table).